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ABSTRACT 

Seventy-four college students participated in a peer 
review assignment, subjects were asked to write a draft of a 
three-page paper, distribute copies to three peer reviewers, revise 
their papers using the resulting feedback from each of the three peer 
reviewers, and then prepare and submit a final paper. Reviews were 
scored for the quality and specificity of feedback. Final papers were 
scored for the degree to which appropriate feedback was accepted. 
Results suggest that peer comments improved writing and that students 
of all ability levels can select and apply appropriate feedback. 
Papers receiving higher grades were more likely to have accepted 
appropriate feedback. Feedback that was judged to be of higher 
quality was more likely to be accepted. While students who wrote 
better papers and made higher course grades gave better feedback, the 
willingness to accept appropriate feedback was unrelated to course 
grade . ( Au t hor/SLD ) 
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Abstract 

Seventy four college students participated in a peer review assignment. 
Reviews were scored for the quality and specificity of feedback. Final papers 
were scored for the degree to which appropriate feedback was accepted. The 
results suggest that peer comments improve writing and that students of all 
ability levels can select and apply appropriate feedback. Papers receiving 
higher grades were more likely to nave accepted appropriate feedback G>.389). 
Feedback judged to be higher quality was more likely to be accepted 0>.38O). 
While students who wrote better papers and made higher course grades gave 
better feedback G>.213 and i*A6$) t willingness to accept appropriate feedback 
was unrelated to course grade (i=.1 18, n.s.). 
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Peer Review: ft Works 

Peer review of written work has been suggested as a useful cooperative 
learning technique in the college classroom (e.g., Beaven, 1977; Bruffee, 1978; 
Damon, 1984; DiPardo & Freedman,1988; Gere, 1987; Graves, 1983; Nystrand, 
1986; Nystrand & Brandt, 1989;Sharan & Shachar, 1988; Slavin, 1983; 
Williams and Colomb, 1990.) Slavin (1983) and others have argued that 
cooperative learning enhances learning and motivation. Vygotsky (1981) and 
Piaget (1959) argue that higher level cognitive functions develop through 
'collective' argument and interaction. Allowing students to critique each others' 
writing also helps develop the notion of 'audience,' which should sensitize 
writers to the reader's perspective. 

Despite this theoretical support for peer review, there has been 
considerable disagreement about the quality and varieties of feedback actually 
provided by students versus teachers. The utility o f peer reviews obviously 
depends on the quality of the feedback and the ability of authors to select and 
apply appropriate versus inappropriate feedback. The scant empirical research 
on the effectiveness of peer review has found both positive (e.g., Nystrand, 
1986: Nystrand, 1989; Kottke, 1988) and negative results (Newkirk, 1984). One 
major concern has been that students will not be able to give effective feedback 
and/or that student authors will not be able to discriminate between appropriate 
and inappropriate feedback. This study addresses this issue. 

Method 

Seventy-four students in a beginning educational psychology class were 
asked to write a draft of a paper (approximately 3 pages), distribute copies to 
three peer reviewers, then revise their papers using the resulting feedback, and 
turn in a final paper. Each author also served as a reviewer for three drafts. 
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Reviewers and authors were encouraged to discuss drafts as well as provide 
written comments. Reviewers could not review drafts of friends. Reviewers had 
about 10 days to do their reviews, and authors had another 10 days to complete 
their final papers. The writers' assignment was to analyze a controversial 
situation, using concepts from the text and lectures. 

In particular, students read a newspaper article on controversial new 
schools designed exclusively for black, inner city boys. They were asked to 
formulate a position regarding the appropriateness of such schools and support 
their opinion based on information and perspectives from the course. 
Instructions included a list of questions authors might want to consider (e.g., 
Which of the theorists that you've read would think this is a good idea? Which 
would think this is a bad idea? In what ways do middle class and inner city 
environments typically differ that have educational impact? Are middle class 
and inner city children often rewarded (and/or punished) for different 
behaviors? Are boys and girls rewarded (and/or punished) for different 
behavio.s? Are African American rewarded (and/or punished) for different 
behaviors compared to other ethnic groups? What are the implications? How 
are motivational patterns likely to differ for middle class versus these young 
men? How does racial prejudice impact on the educational process?) 
Included, too, was a list of concepts which might apply (e.g.. achievement 
motivation, accommodation, assimilation, cognitive development, identity crisis, 
self-concept, values clarification, attribution, autonomy, authoritative teachers, 
early experiences, schemas, social learning, behavior modification, conformity, 
gender identity, selective attention, parenting style, role models, single parent, 
frustration-aggression theory). 
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Students were given specific instructions on how to review, focusing on 
content, structure, clarity and logic rather than spelling or grammar. The training 
for reviewers consisted of an in-class lecture plus a two-page hand-out with 
specific questions. Examples include "What is the main point? Is it clearly 
identified? Is it identified early in the paper? Is it explicit? Is it what you 
expected after having read the first paragraph? What are the two features that 
most need improvement?" Reviewers did not assign grades to papers, but 
wrote comments and if both parties wanted, discussed papers with authors. 

Authors were instructed to consider reviewers' comments carefully, but to 
accept only those that they thought would improve their paper. They were 
assured that the instructor and not the student reviewers would decide tha 
paper's grade. Students were also graded on four exams, quality of the reviews 
they did of others' papers, and several other assignments. 

Reviews were scored independently by two researchers on a 4 point 
scale for quality of feedback and specificity. High quality feedback was 
appropriate, accurate, and helpful; low quality feedback was incorrect or 
irrelevant. Specific feedback applied to a specific phrase or sentence while 
general feedback referred to the overall paper. Final drafts of ail papers were 
first graded (by both researchers) without writing the grade on the paper. The 
grade was based on the extent to which students used concepts from the class 
as well as quality of writing. Then papers were scored on a global three point 
scale for the degree to which the final version had incorporated appropriate 
suggestions from eacn review, in this case, a 3 indicated acceptance of 
appropriate feedback only, a 2 indicated mixed acceptance, and a 1 indicated 
no acceptance or acceptance of inappropriate feedback only. Four scores were 
thus available for each student's paper: grade, acceptance of review #1. 
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acc ep t an ce of raview #2, and accepflance of rewew #3. Each stytiens aSso had 
a soora off tne qtuialty of ttneiir reviews of cttas* papers |a ntaaoBTOjgnn of 12 
points), tresis ftoased on 100 pants), and course grade fbased on 100 points).. 

Resufts 

Tne two raters sta&ed an Kngfn Bevefi off agreement on eacft measure: 
Interrafler re&abiitites tor paper grades, i- 83; for qua% off feedback. r=w9Q„ for 
specfctty. i - .92. and 1!0r acceptance, -88. 

Tne m&siufts findcst© ffltra! peer imew «s effective: 

1 . Better papers resiled tew acceptance of appropriate feedback Cpaper 
grade by acceptance* £-.389.. fi<..0tj}- 

2. Sttwdents ©cxmiltf d&isoriiminste quafifly feedback iimrellevantt feedback: 
nijgn quality ffeectedk was roire SWy to be accepted fa*. 380. jfcc.01)- 

3. As tltie speoftdty of tne feedback (increased,, acceptance itnaeased. ((£— 
.457.flfcc.01) 

4. Sttwdents" course grade was onreliated to theur atolitty to dlsCTtinmnatte use&jiii 
ffirwp imoifTHUJseftuiif ffeedfeack. fcowse grade by apprepnaie accept a nce of 
feedback. i = -118. n.s). 

5. Atitfuoygn conceptually qoaifoty of feedback and speotaty were 
independents p.e, n nigra quality feedback copiid nawe been general! amid Itow 
qwaltity feedback cooUd nave been specie!., tpa&tty and speoifctty of tne 
actyai! feedback were foagfhtiy cofnreEated (&=~.655. jfccOl ). tnat is sOudent 
reviewers gave iwdhi better feedback wcta? titoey ga*e specie ratner ten 
genera? ffeedtoack. 

As nnagrat be expected. qpalsty off tne renews was a function off tne 
stydents" overall perforarance en tne coejrse tfayttoors couirse grade by atuntars 
reviews. £- .465. sk 01 ).. As msigftut aliso be expected, tose students wito© mote 
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better papers were also better reviewers (paper grades by grades on reviews, 
r=*. 213, c< 05). The grades on papers and exams were highly correlated 
559, c<.0t), suggesting that both types of tasks measured similar knowledge 
and skills. 

Discussion 

While quality of the reviews varied and was related to student ability, 
students of all ability levels were able to select and use appropriate feedback. 
While acceptance of feedback was not related to ability, there were some 
students who ignored feedback. It was not possible in this study to determine 
why some students steadfastly refused to accept even very specific, high quality 
feedback. For example, several students referred to the 'state' of Milwaukee 
and in each case, several reviewers pointed out that Milwaukee was a city 
rather than a state. These authors rewrote their papers without making the 
suggested change. Informal discussion with 'resistors' suggested the problem 
often went beyond a writing problem. Some resistors expressed open hostility 
towards their reviewers and the idea that other students had been allowed to 
read their papers. Peer review can obviously be problematic with very 
defensive and hostile students. Group composition may need to be carefully 
monitored to avoid defensiveness. Also, the result showing that quality cf the 
review was related to student ability points to the importance of using student 
ability as a factor in assigning reviewers. Peer review could fail if a student 
happened to be assigned all low performing reviewers and hence did not 
receive any high quality feedback. 

Nystrand (1989) suggests discussion of drafts is one key to improvement 
of drafts, and it may be that discussion also contributes to the acceptance or 
rejection of feedback. In the present study students were not required to hold 
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forma! discussions with their reviewers, but many students reported doing so. I 
suspect from my discussion with students that those who taJked with their 
reviewers were more likely to accept feedback. Clearly research is needed to 
discover more about the conditions under which various students will accept or 
reject appropriate feecfcack. 

The most important finding is that acceptance of feedback improved 
writing. This resuft suggests that even in a targe class without a great deal of 
training, college students' writing can be improved through peer review. The 
results from this study add important information to our understanding of peer 
review as a form of cooperative teaming in the college classroom: Peers can 
give useful feedback and student authors can benefit from their peers* 
comments. 
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